v2.4 to 2.5

Known Issues

Broken project filter before 2.5.15

Argo CD 2.4.0 introduced a breaking API change, renaming the project filter to projects.

Impact to API clients

A similar issue applies to other API clients which communicate with the Argo CD API server via its REST API. If the client uses the project field to filter projects, the filter will not be applied. The failing project filter could have detrimental consequences if, for example, you rely on it to list Applications to be deleted.

Impact to CLI clients

CLI clients older that v2.4.0 rely on client-side filtering and are not impacted by this bug.

How to fix the problem

Upgrade to Argo CD >=2.4.27, >=2.5.15, or >=2.6.6. This version of Argo CD will accept both project and projects as valid filters.

Broken matrix-nested git files generator in 2.5.14

Argo CD 2.5.14 introduced a bug in the matrix-nested git files generator. The bug only applies when the git files generator is the second generator nested under a matrix. For example:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: guestbook
spec:
  generators:
  - matrix:
      generators:
        - clusters: {}
        - git:
            repoURL: https://git.example.com/org/repo.git
            revision: HEAD
            files:
              - path: "defaults/*.yaml"
  template:
    # ...

The nested git files generator will produce no parameters, causing the matrix generator to also produce no parameters. This will cause the ApplicationSet to produce no Applications. If the ApplicationSet controller is configured with the ability to delete applications, it will delete all Applications which were previously created by the ApplicationSet.

To avoid this issue, upgrade directly to >=2.5.15 or >= 2.6.6.

Configure RBAC to account for new applicationsets resource

2.5 introduces a new applicationsets RBAC resource.

When you upgrade to 2.5, RBAC policies with * in the resource field and create, update, delete, get, or * in the action field will automatically grant the applicationsets privilege.

To avoid granting the new privilege, replace the existing policy with a list of new policies explicitly listing the old resources.

Example

Old:

p, role:org-admin, *, create, *, allow

New:

p, role:org-admin, clusters,     create, *, allow
p, role:org-admin, projects,     create, *, allow
p, role:org-admin, applications, create, *, allow
p, role:org-admin, repositories, create, *, allow
p, role:org-admin, certificates, create, *, allow
p, role:org-admin, accounts,     create, *, allow
p, role:org-admin, gpgkeys,      create, *, allow
p, role:org-admin, exec,         create, *, allow

(Note that applicationsets is missing from the list, to preserve pre-2.5 permissions.)

argocd-cm plugins (CMPs) are deprecated

Starting with Argo CD v2.5, installing config management plugins (CMPs) via the argocd-cm ConfigMap is deprecated. Support will be removed in v2.7.

You can continue to use the plugins by installing them as sidecars on the repo-server Deployment.

Sidecar plugins are significantly more secure. Plugin code runs in its own container with an almost completely-isolated filesystem. If an attacker compromises a plugin, the attacker’s ability to cause harm is significantly mitigated.

To determine whether argocd-cm plugins are still in use, scan your argocd-repo-server and argocd-server logs for the following message:

argocd-cm plugins are deprecated, and support will be removed in v2.6. Upgrade your plugin to be installed via sidecar. https://argo-cd.readthedocs.io/en/stable/user-guide/config-management-plugins/

NOTE: removal of argocd-cm plugin support was delayed to v2.7. Update your logs scan to use v2.7 instead of v2.6.

If you run argocd app list as admin, the list of Applications using deprecated plugins will be logged as a warning.

Dex server TLS configuration

In order to secure the communications between the dex server and the Argo CD API server, TLS is now enabled by default on the dex server.

By default, without configuration, the dex server will generate a self-signed certificate upon startup. However, we recommend that users configure their own TLS certificate using the argocd-dex-server-tls secret. Please refer to the TLS configuration guide for more information.

Invalid users.session.duration values now fall back to 24h

Before v2.5, an invalid users.session.duration value in argocd-cm would 1) log a warning and 2) result in user sessions having no duration limit.

Starting with v2.5, invalid duration values will fall back to the default value of 24 hours with a warning.

Out-of-bounds symlinks now blocked at fetch

There have been several path traversal and identification vulnerabilities disclosed in the past related to symlinks. To help prevent any further vulnerabilities, we now scan all repositories and Helm charts for out of bounds symlinks at the time they are fetched and block further processing if they are found.

An out-of-bounds symlink is defined as any symlink that leaves the root of the Git repository or Helm chart, even if the final target is within the root.

If an out of bounds symlink is found, a warning will be printed to the repo server console and an error will be shown in the UI or CLI.

Below is an example directory structure showing valid symlinks and invalid symlinks.

chart
├── Chart.yaml
├── values
│   └── values.yaml
├── bad-link.yaml   -> ../out-of-bounds.yaml       # Blocked
├── bad-link-2.yaml -> ../chart/values/values.yaml # Blocked because it leaves the root
├── bad-link-3.yaml -> /absolute/link.yaml         # Blocked
└── good-link.yaml  -> values/values.yaml          # OK

If you rely on out of bounds symlinks, this check can be disabled one of three ways:

1. The --allow-oob-symlinks argument on the repo server.
2. The reposerver.allow.oob.symlinks key if you are using argocd-cmd-params-cm
3. Directly setting ARGOCD_REPO_SERVER_ALLOW_OOB_SYMLINKS environment variable on the repo server.

It is strongly recommended to leave this check enabled. Disabling the check will not allow all out-of-bounds symlinks. Those will still be blocked for things like values files in Helm charts, but symlinks which are not explicitly blocked by other checks will be allowed.

Deprecated client-side manifest diffs

When using argocd app diff --local, code from the repo server is run on the user’s machine in order to locally generate manifests for comparing against the live manifests of an app. However, this requires that the necessary tools (Helm, Kustomize, etc) are installed with the correct versions. Even worse, it does not support Config Management Plugins (CMPs) whatsoever.

In order to support CMPs and reduce local requirements, we have implemented server-side generation of local manifests via the --server-side-generate argument. For example, argocd app diff --local repoDir --server-side-generate will upload the contents of repoDir to the repo server and run your manifest generation pipeline against it, the same as it would for a Git repo.

In v2.7, the --server-side-generate argument will become the default, and client-side generation will be supported as an alternative.

Warning

The semantics of where Argo will start generating manifests within a repo has changed between client-side and server-side generation. With client-side generation, the application’s path (spec.source.path) was ignored and the value of --local-repo-root was effectively used (by default / relative to --local).

For example, given an application that has an application path of /manifests, you would have had to run argocd app diff --local yourRepo/manifests. This behavior did not match the repo server’s process of downloading the full repo/chart and then beginning generation in the path specified in the application manifest.

When switching to server-side generation, --local should point to the root of your repo without including your spec.source.path. This is especially important to keep in mind when --server-side-generate becomes the default in v2.7. Existing scripts utilizing diff --local may break in v2.7 if spec.source.path was not /.

Upgraded Kustomize Version

The bundled Kustomize version has been upgraded from 4.4.1 to 4.5.7.

Upgraded Helm Version

Note that bundled Helm version has been upgraded from 3.9.0 to 3.10.1.

Upgraded HAProxy version

The HAProxy version in the HA manifests has been upgraded from 2.0.25 to 2.6.2. To read about the changes/improvements, see the HAProxy major release announcements (2.1.0, 2.2.0, 2.3.0, 2.4.0, 2.5.0, and 2.6.0.

Logs RBAC enforcement will remain opt-in

This note is just for clarity. No action is required.

We expected to enable logs RBAC enforcement by default in 2.5. We have decided not to do that in the 2.x series due to disruption for users of Project Roles.

argocd app create for old CLI versions fails with API version >=2.5.16

Starting with Argo CD 2.5.16, the API returns PermissionDenied instead of NotFound for Application GET requests if the Application does not exist.

The Argo CD CLI before versions starting with version 2.5.0-rc1 and before versions 2.5.16 and 2.6.7 does a GET request before the POST request in argocd app create. The command does not gracefully handle the PermissionDenied response and will therefore fail to create/update the Application.

To solve the issue, upgrade the CLI to at least 2.5.16, or 2.6.7.

CLIs older than 2.5.0-rc1 are unaffected.

Golang upgrade in 2.5.20

In 2.5.20, we upgrade the Golang version used to build Argo CD from 1.18 to 1.19. If you use Argo CD as a library, you may need to upgrade your Go version.